Vital signs of a person, for example the heart rate (HR), the respiration rate (RR) or the arterial blood oxygen saturation (SpO2), serve as indicators of the current health state of a person and as powerful predictors of serious medical events. For this reason, vital signs are extensively monitored in inpatient and outpatient care settings, at home or in further health, leisure and fitness settings.
One way of measuring vital signs is plethysmography. Plethysmography generally refers to the measurement of volume changes of an organ or a body part and in particular to the detection of volume changes due to a cardio-vascular pulse wave traveling through the body of a subject with every heartbeat.
Photoplethysmography (PPG) is an optical measurement technique that evaluates a time-variant change of light reflectance or transmission of an area or volume of interest. PPG is based on the principle that blood absorbs light more than surrounding tissue, so variations in blood volume with every heart beat affect transmission or reflectance correspondingly. Besides information about the heart rate, a PPG waveform can comprise information attributable to further physiological phenomena such as the respiration. By evaluating the transmittance and/or reflectivity at different wavelengths (typically red and infrared), the blood oxygen saturation can be determined.
Recently, non-contact, remote PPG (rPPG) devices (also called camera rPPG devices) for unobtrusive measurements have been introduced. Remote PPG utilizes light sources or, in general radiation sources, disposed remotely from the subject of interest. Similarly, also a detector, e.g., a camera or a photo detector, can be disposed remotely from the subject of interest. Therefore, remote photoplethysmographic systems and devices are considered unobtrusive and well suited for medical as well as non-medical everyday applications. This technology particularly has distinct advantages for patients with extreme skin sensitivity requiring vital signs monitoring such as Neonatal Intensive Care Unit (NICU) patients with extremely fragile skin or premature babies.
Verkruysse et al., “Remote plethysmographic imaging using ambient light”, Optics Express, 16(26), 22 Dec. 2008, pp. 21434-21445 demonstrates that photoplethysmographic signals can be measured remotely using ambient light and a conventional consumer level video camera, using red, green and blue color channels.
Wieringa, et al., “Contactless Multiple Wavelength Photoplethysmographic Imaging: A First Step Toward “SpO2 Camera” Technology”, Ann. Biomed. Eng. 33, 1034-1041 (2005), discloses a remote PPG system for contactless imaging of arterial oxygen saturation in tissue based upon the measurement of plethysmographic signals at different wavelengths. The system comprises a monochrome CMOS-camera and a light source with LEDs of three different wavelengths. The camera sequentially acquires three movies of the subject at the three different wavelengths. The pulse rate can be determined from a movie at a single wavelength, whereas at least two movies at different wavelengths are required for determining the oxygen saturation. The measurements are performed in a darkroom, using only one wavelength at a time.
Apart from the advantage of being fully contactless, cameras (generally called imaging devices) provide 2D information, which allows for a multi-spot and large area measurement, and often contains additional context information. Unlike with contact sensors, which rely on the correct placement on a specific measurement point/area, the regions used to measure pulse signals using rPPG technology are determined from the actual image. Therefore, accurate detection of skin areas, reliable under any illumination conditions becomes a crucial part in the processing chain of a camera-based rPPG device and method used for camera-based vital signs monitoring.
Currently, there are two main approaches known for reliable detection and tracking of a skin areas.
One approach is based on skin color (RGB-based) detection and segmentation. Methods according to this approach are fast in both detection and tracking of areas with skin color. However, they are not robust to changes of ambient light color, which will change the color of light reflected from a skin area, and are not able to detect skin areas under low illumination conditions or in darkness. Moreover, such methods cannot always differentiate a skin from other objects with the same color.
Another approach is based on extracted PPG signals (PPG-based). Methods according to this approach are more robust in differentiating real skin areas and areas of other object of the same skin color. This approach can be used also to segment the skin areas, which have stronger PPG signal (the most periodic signal). However, the reliability of the approach depends on the robustness of PPG signal extractions, thus it is impacted by motion of a subject and the blood perfusion level. Therefore, if a pulse signal is not periodic or is weak, a camera-based system will have difficulties to detect the segment the skin areas. Moreover, the approach is also computationally expensive.
It should be noted that the detection of skin area is not only of interest in the field of vital signs detection based on the rPPG technology, but also in other technical fields, e.g. in remote gaming applications using camera technology to recognize gestures of the player, face detection, security (robust detection of a person using surveillance cameras and detection of a person wearing a mask or distinguishing real faces from a realistic mask in a camera registration), etc.
A problem with surveillance camera images is that it is difficult to distinguish between a face and a mask or a picture of a face. A problem with camera-based vital signs monitoring is that the vital signs can only be extracted from living tissue, usually some skin-area, but currently this area is most often manually selected from the video and consequently tracked, or one has to rely on a face detector that typically fails on side views of a face, and certainly does not give anything useful for non-facial skin in the video. Hence, there is still a need for ways to reliably, accurately and quickly find every useful skin surface.